Implement non-blocking provider setup with gated tool stubs#28
Merged
Conversation
Provider setup (git clone + build, per-requirement pip install, and setup_commands such as `npx playwright install chrome`) previously ran synchronously and sequentially at import, before mcp.run() — so the MCP endpoint accepted no connections until the slowest provider finished installing. Now every provider's tools are registered up front as gated stubs built from the YAML alone, and the slow setup runs on a background daemon thread started in __main__. The MCP server (and UI) come up immediately; a tool whose provider is still installing returns a structured retry directive (`status: "initializing"`, `retry_after_seconds`) instead of failing or hanging, and the same registered tool transparently delegates to the real handler once setup completes. A provider whose handlers fail to build surfaces `status: "failed"` at call time. - New provider_status.py holds per-provider readiness state shared in-process (added to the Dockerfile COPY line; guarded by test_dockerfile.py). - Extracted build_tool_handlers() from register_provider() so the real handlers can be resolved on the background thread; register_provider and bootstrap_provider keep their existing behavior (synchronous opt-out path). - Knobs: MCPPROXY_BACKGROUND_SETUP (default 1) and MCPPROXY_INIT_RETRY_SECONDS (default 15). - Pin PIP_CACHE_DIR / UV_CACHE_DIR to the persisted /root/.cache volume. - Document the behavior in the README and add tests for the gated flow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RX32pG1y9CvXd2VXHPzM8q
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements non-blocking provider setup for mcpproxy. Instead of blocking the MCP server startup while installing dependencies for all providers, the server now registers tools immediately as gated stubs and runs setup in the background. Tools whose providers are still initializing return a structured retry directive rather than failing.
Key Changes
New
provider_status.pymodule: Tracks per-provider readiness state (PENDING/READY/FAILED) and holds real tool handlers once setup completes. Shared between server.py's gated handlers and the frontend UI.Refactored
register_provider()inserver.py: Extracted handler-building logic into a newbuild_tool_handlers()function that is setup-independent. This allows handlers to be built either synchronously (old behavior) or after background setup completes (new behavior).New gating mechanism:
register_gated_provider(): Registers tools as stubs built from YAML alone, without running setup or executing code._make_gate_handler(): Creates a handler that dispatches based on provider state — returns retry directive while PENDING, delegates to real handler when READY, or surfaces error when FAILED._resolve_provider(): Runs setup and builds real handlers on a background thread, flipping provider state from PENDING to READY/FAILED.Configuration knobs:
MCPPROXY_BACKGROUND_SETUP(default:1): Set to0to use the old synchronous behavior.MCPPROXY_INIT_RETRY_SECONDS(default:15): Seconds advertised in retry directives.Startup flow: When background setup is enabled, all providers are registered as gated stubs immediately, then a background thread runs
_background_bootstrap()to sequentially resolve each provider's setup and handlers.Updated documentation: README now explains non-blocking startup, the retry directive format, and the configuration options.
Implementation Details
MCPPROXY_BACKGROUND_SETUP=0restores the original synchronous behavior.https://claude.ai/code/session_01RX32pG1y9CvXd2VXHPzM8q